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ABSTRACT 

An instructional approach for conceptualizing the five 
domains of. the instructional technology knowledge based defined by the 
Association for Educational Communications and Technology (AECT) Initial and 
Advanced Program Guidelines-design, development, utilization, management and 
assessment-that were required to be integrated into professional electronic 
portfolios developed by educational technology degree candidates is examined. 
An in depth look at the process is provided through quantitative and 
qualitative data about the students* perceptions of the utility of the AECT 
domains for communicating their professionalism in their portfolios. The 
process is further examined through the lenses provided by five departmental, 
faculty members* qualitative assessment of the students* portfolios and the 
students* presentations of these artifacts. Results of this preliminary study 
indicate that although the five domains tend to be valued for their 
usefulness in designing to express emerging professionalism, unless required, 
most would not employ them. Once applied, however, most would retain this 
framework. Additional suggestions for modifying the portfolio design’ and 
development process from initiation in a Web-based course towards the 
beginning of the students* course of studies through to the final 
presentation at the conclusion of their program are offered. The increased 
emphasis on portfolio development throughout education, the growth of Web- 
based design, and the acceptance of the AECT Guidelines for candidates in 
both Basic Media Technology in Teacher Education and Advanced Programs in 
Educational Communications and Technology by the Council for the 
Accreditation of Teacher Education suggest continued study. Such study is 
especially warranted in terms of the potential for providing learner guidance 
relative to this area for students enrolled in educational technology 
programs. (Contains 12 references.) (Author /AEF) 
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^ ' Abstract 

■ An instructional approach for conceptualizing the five domains of the instructional technology knowledge base defined by 

AECT Initial and Advanced Program Guidelines —design, development, utilization, management and assessment— that were 
required to be integrated into professional electronic portfolios developed by educational technology degree candidates is 
examined. An in depth look of the process is provided through quantitative and qualitative data about the students* perceptions of 
the utility of the AECT domains for communicating their professionalism in their portfolios. The process is further examined 
through the lenses provided by five departmental faculty members* qualitative assessments of the students* portfolios and the 
students* presentations of these artifacts. Results of this preliminary study indicate that although the five domains tend to be 
valued for their usefulness in designing to express emerging professionalism, unless required, most would not employ them. 
Once applied, however, most would retain this framework. Additional suggestions for modifying the portfolio design aid 
development process from initiation in a web-based course towards the beginning of the students* course of studies through to the 
final presentation at the conclusion of their program are offered. The increased emphasis on portfolio development throughout 
education, the growth of web-based design, and the acceptance of the AECT Guidelines for candidates in both Basic Media 
Technology in Teacher Education and Advanced Programs in Educational Communications and Technology by the Council for 
the Accreditation of Teacher Education (NCATE) suggest continued study. Such study is especially warranted in terms of the 
potential for providing learner guidance relative to this area for students enrolled in educational technology programs. 



Introduction 

The Accreditation Committee of Association for Educational Communications and Technology (AECT) provides Folio 
Review Guidelines related to educational technology. The guidelines include Basic Guidelines for Media Technology in Teacher 
Education, as well as for Advanced Programs in Educational Communications and Technology (Association for Educational 
Communications and Technology, 2000) that are accepted by the National Council for the Accreditation of Teacher Education 
(NCATE). These guidelines provide a systematic means for internal and external program reviews at colleges and universities 
throughout the United States. 



Standards in Education 

In this twenty -first century such an emphasis on standards for education is not uncommon. Examples can be found relative to 
overall professionalism for teachers (National Board for Professional Teaching Standards, 2001), specific Standards for 
Technology in Teacher Preparation (International Society for Technology in Education, 2001), and specific content areas, such as 
standards for mathematics (National Council for the Teachers of Mathematics, n.d.). 

Standards in Educational Technology 

In the field of educational technology, candidates in both entry level and advanced programs are expected to acquire 
knowledge and skill within five main areas of the instructional technology knowledge base in domains of: design, development, 
utilization, management and assessment. Within advanced programs, candidates should demonstrate minimal competencies 
within each area along with advanced capability in aspects specifically emphasized by the program at their institutions. 
Web-Based Delivery of Instruction and Professional Portfolio Development. 

Additionally, two other areas that impact educational technology majors are web-based delivery of instruction and 
professional portfolio development. These students may acquire some of their capabilities through web-based coursework. 
Educational technology majors, as current and future teachers or trainers, may be expected to teach by means of the World Wide 
Web (WWW). Experience with this mode of learning is essential for their future distance learners and for their colleagues when 
they function as consultants. Furthermore, these majors are increasingly expected to demonstrate their knowledge and skills at the 
completion of their program of studies in electronic portfolios. 



cs 

o 




Professional Electronic Portfolios 

Professional electronic portfolios would appear to be an integral part of the core concepts of post-modernism that Solomon 
(2000) relates to instructional technology. Professional portfolio development supports: 1) respect for differences and multiple 
explanations through inherent nonlinear design; 2) construction of personal views by prospective employers and others through 
text, audio, and visual information about the candidate; 3) critical evaluation as candidates assess themselves at the design phase; 
and 4) expression, to some extent, of the complexity of the system of educational technology as hypermedia links to design. 
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development, utilization, management, and evaluation aspects and to candidates’ professionalism and theoretical constructs of the 
field. Additionally, development of Web-based electronic portfolios is growing at all levels of education (Sanders, 2000). 

Web-Based Course. Program of Studies, and Wet>Based Production 

At a department of educational technology at one Pacific-based university, graduate students participate in a totally Web- 
based course on newer technologies. Typically, they enroll in this course in their first or second semester in the program. The 
Web-based delivery mode of this course with its multilevel, hypertext structure and design for interactivity with the learners by 
synchronous and asynchronous communications, text, audio, and graphic display (Chu 2000) offers a supportive context for core 
concepts of postmodernism. 

Along with earning 39 graduate level credits in required and elective courses, completing 6 credits in prerequisite courses, 
engaging in both a collaborative and individual practicum, and carrying out a comprehensive master’s project, these students were 
recently required to also produce a professional electronic portfolio during their program of studies. Introduction to this portfolio 
process was incorporated in the required Web-based course that addresses basic concepts of instructional design and multimedia 
authoring. Typically, students enroll in this course during their first or second semester of study. In following this newer 
departmental process for developing this Web-based multimedia perspective about themselves, they continue the process 
throughout their program. As currently designed, after the initial course, consultation with peers and advisors about their 
continued development of their portfolios occurs primarily when initiated by the student. During a culminating course, they do 
have the opportunity to critique each other's work prior to formally presenting their completed portfolios to the faculty. 

When this project was first instituted, students studied multimedia design principles (Park and Hannafin, 1993) and addressed 
general guidelines for content broadly described to encourage individuality of expression. At the conclusion of the course, initial 
portfolios were assessed in terms of effective design and plan for content coverage. An initial investigation of the students’ work 
indicated that, although they demonstrated competencies in all five domains that define an educational technologist, they did not 
tend to overtly conceptualize their expertise within this framework. Many communicated their capabilities in terms of courses, 
which may or may not be meaningful to future employers (Sherry, 2000). 

Problem Statement 

Although faculty in educational technology programs, particularly those members engaged in the NCATE process, may 
become adept at describing their programs and their candidates’ capabilities in terms of these domains, how might their students 
begin to conceptualize their leaning within this framework? Presenting their competencies in this manner could offer potential 
employers and their graduate faculty a consistent, professional view of their expertise in these critical areas in relation to 
standards set by a professional association. 

As these findings emerged, departmental faculty members decided to disseminate information about the domains and the 
framework for the portfolios as the students were developing their initial portfolios in the Web-based course. They followed up 
with a print -based reminder as the students advanced toward completion of their portfolios prior to graduation. These students 
earn the required minimum of 39 credits in courses in areas specific to these domains and in specific related aspects of 
educational technology. As students who are used to developing and analyzing instructional designs in a structured manner, it 
became essential to evaluate their perceptions of the utility of, and type of integration of, this structured framework for 
expressing their nascent professionalism. 

Methodology 

An investigation was, thus, undertaken as a preliminary evaluation study. 

Ethical Practice and Participants 

This examination, construed as standard educational practice, received the anticipated exempt classification after review by 
the Institutional Review Board Committee for the Protection of Human Subjects at the university where the study was conducted. 
In compliance with that process, all participants were fully apprised in writing of the nature of the study, their option for 
voluntary participation, and their option to participate in having examples of their work displayed as results of the study were 
disseminated. Of the eight students who completed portfolios prior to graduation, seven agreed to participate in the study. The 
resulting 88% return rate, represents five of the six females and both of the males; an acceptable rate to provide an initial 
perspective of the perceived usefulness of the five domains for communicating professionalism as educational technologists 
through the medium of electronic portfolios. 

Data Gathering 

Data are based on quantitative and qualitative responses from, and the actual portfolios created by, participants and on 
qualitative data from their faculty members. 

Survey Instrument . 

As no published survey was identified that addressed the questions being studied, a written survey was developed. Questions 
designed to capture demographic data about gender, employment, planned portfolio usage, experience producing portfolios prior 
to creating these professional portfolios, and awareness of the five domains of an educational technologist were included in the 
survey. The students’ views of the usefulness of the five domains were operationalized in terms of five constructs. The five are 
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the utility of employing the domains in their portfolios for communicating: 1) the students' knowledge, 2) the students’ skills, 3) 
the range of the students' professionalism, 4) the contribution the domains made to the students’ descriptions, and 5) the students’ 
intent to retain the domains in their portfolios after graduation. Their reactions to these five aspects were captured on a ten item 
survey that has a negatively and positively worded item for each aspect. 

The Loyd and Gressard (1986) Computer Attitude Survey consisting of 40 positively and negatively worded items about 
attitudes toward computers served to guide the construction of these types of questions. For example, the two items about 
perceptions of skills appear as, "The 5 domains are useful in my electronic portfolio in describing my skills in the field of 
educational technology" and "Mv skills can be described better if descriptors other than the five domains are used." A graduate 
student in the program also offered one-on-one feedback about the survey during its construction. 

Students indicated their degree of agreement on a four point Likert -like scale that ranged from "strongly agree" to "strongly 
disagree". In addition, five open-ended questions were developed to obtain input about the type of examples that students' 
selected for their portfolios vdien expressing their performances in each of the five domains of an educational technologist. A 
final broad, open-ended question sought additional comments. 

Artifacts . 

The students' electronic portfolios also served as a means of verifying application of the domains. 

Faculty Responses . 

The five full-time departmental faculty members also provided oral input after viewing the students' formal presentations of 
their portfolios. The relatively small size of the department (5 full-time faculty members and c. 50 master’s degree candidates) did 
not permit the separation of the investigator as a participant. 

Data Analysis 

During analysis, an emphasis on the more objective data — the quantitative responses from the students — ^and validation of the 
faculty members* judgements expressed as emergent themes about the students’ portfolios and presentations were confirmed by 
all the faculty members. This approach was taken to mitigate to some degree any potential for bias that might occur from the 
close association between the primary investigator and resulting interpretations of the data. Despite the small numbers of 
participants, the multiple sources of data collected in this study do have potential to offer an initial view of a process that others 
may wish to employ to some degree as they engage in similar efforts. 

Quantitative Data 

Quantitative data were analyzed in terms of frequency distributions. From those analyses the picture that appears shows that 
among the seven respondents who provided usable data, two were unemployed prior to graduation, four were working as 
graduate assistants in a technology support capacity, and one was employed in a network security role. Five had had no 
experience producing any type of portfolio prior to initiating work on their professional electronic portfolios for their master's 
program. All of the participants indicated that they only become aware of the five domains of an educational technologist as they 
completed their electronic portfolios. 

To more readily express the sentiments of the limited number of participants, the "strongly" and "slightly" agree categories 
for both agreement and disagreement were collapsed into two categories of agreement and disagreement. 

As indicated in Figure 1 , the five domains are viewed a contributing to the communication of knowledge and skills as an 
educational technologist by six of the seven respondents for the former aspect and all of the students for the latter one. 
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Help Hinder Help Are Not as Effective 



as Other Means 

Knowledge Description Skills Description 



Figure 1 . Perceptions of the usefulness of the five domains of an educational technologist when conceptualizing professionalism 
as an educational technologist in a professional electronic portfolio. 

Despite the positive regard for applying the domains to describe skills, negativity predominates (n=4) in regard to the domains 
being used for skill description rather than some other means. 

Figure 2 reflects students' responses to the picture of the domains, and the perceived contribution to professionalism, as well 
as, to confusion during use. For expressing overall professionalism, all but one of the respondents view the domains as being 
helpful and not hindering such a depiction. Most (n=5) view the domains as contributing to depicting their range of capabilities. 
The same number, however, do agree that during production the domains are confusing to use. 
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Figure 2 . Perceptions of the usefulness of the five domains of an educational technologist when conceptualizing one's range of 
professionalism and as an educational technologist and of the contribution that the domains made to this picture in a 
professional electronic portfolio. 

That same number of students state they would not employ the domains if they were not required to do so, although five of 
them do plan to retain usage of the domains (see Figure 3). 
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Fipire 3 . Perceptions of plans to retain the five domains of an educational technologist to communicate professionalism as an 
educational in a professional electronic portfolio. 

Studeuts* Qualitative Data 

Six of the seven respondents completed the open-ended section of the survey that related to the manner in which they 
conceptualized the examples they selected to represent their professionalism for each of the five domains. Half briefly referenced 
their selections. Each domain, for example, is expressed in the following manner; for design, "Grant from ETEC 600;" for 
development, "ETEC 603 Designing a paper-based module;" for management, "ETEC 650 assignments;" for utilization, "ETEC 
662 Planning a computer lab (networking)"; and for evaluation, "final project" [master's study]. Of those respondents who 
expressed their choices in this manner, they referred specifically to nine courses, five of which are required courses in the 
master’s degree program. 

The other three respondents offered brief narratives about their knowledge and skills, rather than referencing a course alpha 
and number. For example, in relation to the domain of design, one student wrote, "Print -based instructional module — surfing 
lessons" This instructional module was designed following the Dick & Carey instructional design model." A similar example, in 
relation to the domain of management is expressed as, 

A visit to UHM video (on Products Page) based on Mintzberg’s (1971) principles of mgmt. I was the director of the video. I 
served as the liaison between the various actors groups for resources & communication. I was resource allocator & the negotiator 
in that I made sure the project was recorded, edited, & turned in on time. 

Analysis of Portfolios 

A review of the portfolios themselves by all the departmental faculty members indicates that all the students completed their 
portfolios, addressing design and content requirements to a degree deemed acceptable for this newer departmental requirement. 

Comparison of Faculty Members’ and Students* Oualitarive Data 

Qualitative data, the students' responses to the open-ended questions, their actual portfolios, and the faculty members' oral 
comments about the portfolios were analyzed in terms of emergent themes. 

Emergent themes during the analysis of the open-ended comments offered by both the students and the faculty members 
indicate that, although there is some convergence in regard to the issues, students' comments relate primarily to learner guidance 
about the domains. Specifically, they mention the need to provide information about the domains at the beginning of the process 
with details about the domains as they develop their portfolios. There are suggestions related to the format that the guidance 
could assume, such as, a brochure about the domains and examples of portfolios that integrate explicit domain-based information. 

To some extent the feedback from the faculty members reflects the students' ideas. The faculty members' note the need for 
the candidates to be clear and overt in explanations about the domains in their electronic portfolios. To achieve such ends and 
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overall professionalism in portfolios, the faculty members cite a need for a more structured ongoing peer and faculty feedback 
process during the two plus years of design and development. 

In terms of professionalism, the faculty members state that such posture needs to be emphasized in the design and selection 
of content. They explicitly note this situation in terms of the degree of emphasis given to work-related information over personal 
data; selection of concise exemplars; use of sophisticated design elements, including decisions about selecting appropriate 
graphics; and increased focus on the candidates' future worth. Formatting is also mentioned with recommendations for the 
inclusion of alt tags and pdf files to ensure that awareness of ADA and printing concerns are conveyed. 

Of particular note is a reminder to address privacy issues in terms of self and others, such as considering the amount of 
personal information to make readily available on the WWW and obscuring references to specific individuals, or educational 
sites, that might occur in samples included in the portfolios. They also call for developing a way for the students to communicate 
the rich content of these portfolios in a reasonable timeframe, while ensuring professionalism during presentation. 

Discussion 

Despite introducing the concept of analyzing professional knowledge and ^ills and overall range of professionalism in 
relation to the five domains of their field toward the completion of the portfolio process, it is encouraging to note the students’ 
overall positive reaction to the utility of that framework. The process of sy nthesis required of them, rather than analysis, can be a 
more challenging intellectual skill. This cognitive demand may be reflected to some extent in the students’ mixed feelings about 
the contribution that the domains make in conveying the range of the students' capabilities. 

Perceptions of Domains 

Although they primarily agreed (n=5) that the domains did contribute to this picture, the same number also noted that during 
production the domains were confusing to use. Their comments about the need for additional guidance in regard to the domains, 
particularly their request for brochures and examples from the portfolios of others, offer support for providing additional 
information for future learners both about the domains themselves and about applying them to interpret and communicate their 
capabilities. 

At first it might appear that the students do not value the domains as descriptors, as the majority of the students (n=5) indicate 
that they would not employ them unless required to do. Their acknowledgement (n=5), however, that they plan to retain the 
domains as descriptors during revisions appears to reflect the worth which they apparently are beginning to place on the domains. 
A review of the final versions of the portfolios at graduation, when minor revisions were made after the formal presentations also 
supports this contention. 

Expanded descriptions provided by three of the students in regard to addressing each of the domains in their portfolios, as 
opposed to the other students’ succinct references to course alpha and numbers, suggest a deeper integration of the concepts by 
the former. The reason, however, is not evident from data gathered in this study. 

Process Issues 

Instructional design issues, thus, are the predominant concerns identified by responses to the survey, in the actual portfolios, 
and in the faculty members’ evaluations. Contextual issues related to resources — hardware and software accessibility — did not 
appear in any comments other than a reminder from the faculty members about msuring that the required CD-ROM copy of the 
portfolios continue to be made to ensure that the students' work not be lost in "cyberspace." The absence of concerns about 
hardware and software where not surprising. As majors in an educational technology program, personal ownership of a computer 
is expected. Additionally, generous, successful grant funding has resulted in a relatively high-end technology laboratory being 
available not only for these majors, but for all in the college of education. 



Implications 

The students do offer perspectives that indicate their awareness of, as well as the usefulness of, incorporating the five 
domains into their portfolios, as well as a variation in the level of conceptualizing domain usage. As professionals in the field of 
educational technology, planning for effective transfer of critical learning experiences is an essential part of their learning. The 
results of this preliminary investigation about graduate level educational technology students' perspectives of engaging in the 
electronic portfolio process and their depictions of themselves as designers, developers, users, managers, and evaluators in terms 
of these five domains suggest future directions for faculty members in similar situations. 

Incidental information related to candidates’ portfolio development offers a glimpse of the potential connection that might 
result from the students’ efforts in designing and developing their portfolios. Based on a recent summary of Kirkpatrick's classic 
Four-Level Evaluation Model from the late 1950’s (2001), at Level One, Reaction, the participating students hold predominantly 
positive attitudes toward the utility the domain-centered framework provides. At Level Two, Learning, they appear capable of 
communicating their knowled^ and skills in relation to the domains. A slight suggestion of how Level Three, Behavior on the 
Job, is offered in terms of most students indicating that they plan to retain the domains as descriptors in future revisions of their 
portfolios. It may be that they are planning ongoing usage in future work situations. It is nearly impossible to determine in any 
substantive manner insights about such portfolio usage in terms of Kirkpatrick's evaluation Level Four, Results, as it refers to 
impact on the job. Preliminary, incidental data do, however, offer a very faint preliminary sketch. One student nearing 
graduation, who produced a portfolio before the requirement to overtly address the domains existed, received two university- level 
job offers within three days of posting his portfolio on the WWW. This outcome resulted despite his avoidance of posting his 
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work on a job recruitment site. A recent student, who incorporated the domains in her portfolio and obtained a position as a 
lecturer at a two-year college, was told after being selected that her portfolio contributed to the decision to hire her. 

The mere suggestion of such impact makes continuing efforts in refining the professional electronic portfolio process, 
particularly for educational technology majors who are expected to communicate their professionalism using newer technologies, 
essential. It is particularly critical as the portfolio process, in varying formats, gains increasing attention for pre- and in-service 
educators in all fields (Campbell, Cignetti, Melenyzer, Nettles, & Wyman, Jr., 2001 ; Sanders, 2000). 

Additionally, scaffolding candidates’ professional development is an important theme for virtual universities (Collis, 1997). It 
may well be that an electronic portfolio development process wit hin the WWW environment, one that is continually scrutinized 
in terms of the process and output can provide such support for educational technology candidates. Future studies conducted by 
investigators at institutions of higher education in the United States, as well as ones overseas, that offer graduate programs in 
educational technology would seem warranted to determine the ultimate value of this intended exemplar of professionalism. 
Given the time and effort that both faculty and students expend on the p rocess it is critical that such studies occur. 
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